Optical filter, light detecting device, and light detecting system

ABSTRACT

An optical filter includes a filter array including filters two-dimensionally arrayed and a band-pass filter. The filters includes first and second filters. A transmission spectrum of each of the first and second filters has local maximum values of transmittance at three or more wavelengths included in a first wavelength region. The band-pass filter passes light in a second wavelength region including two or more wavelengths of the three or more wavelengths and not including one or more wavelengths of the three or more wavelengths. The filter array and the band-pass filter are disposed so that (a) the band-pass filter is located on an optical path of light that passes through the first and second filters or (b) the first and second filters are located on an optical path of light that passes through the band-pass filter.

BACKGROUND 1. Technical Field

The present disclosure relates to an optical filter, a light detectingdevice, and a light detecting system.

2. Description of the Related Art

Utilizing spectral information of a large number of bands, for example,several tens of bands, each of which is a narrowband, makes it possibleto determine detailed properties of a target object, the determinationhaving been impossible with conventional RGB images. Cameras thatacquire such multi-wavelength information are called “hyperspectralcameras”. For example, as disclosed in U.S. Patent ApplicationPublication No. 2016/138975, U.S. Pat. Nos. 7,907,340 and 9,929,206,Japanese Unexamined Patent Application Publication (Translation of PCTApplication) No. 2013-512445, and Japanese Unexamined Patent ApplicationPublication (Translation of PCT Application) No. 2015-501432, thehyperspectral cameras are utilized in various fields for foodinspection, living-body examination, drug development, mineral componentanalysis, and so on.

SUMMARY

In one general aspect, the techniques disclosed here feature an opticalfilter including: a filter array including a plurality of filterstwo-dimensionally arrayed and a band-pass filter. The plurality offilters includes a first filter and a second filter. A transmissionspectrum of each of the first filter and the second filter has localmaximum values of transmittance at three or more wavelengths included ina first wavelength region. The band-pass filter passes light in a secondwavelength region including two or more wavelengths of the three or morewavelengths and not including one or more wavelengths of the three ormore wavelengths. The filter array and the band-pass filter are disposedso that (a) the band-pass filter is located on an optical path oftransmitted light that passes through the first filter and the secondfilter or (b) the first filter and the second filter are located on anoptical path of transmitted light that passes through the band-passfilter.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically showing a light detecting system in anexemplary embodiment;

FIG. 2A is a view schematically showing an example of a filter array;

FIG. 2B is a view showing one example of spatial distributions oftransmittances of light in respective wavelength regions in a targetwavelength region;

FIG. 2C is a graph showing an example of a transmission spectrum of oneof two areas included in a plurality of areas in the filter array shownin FIG. 2A;

FIG. 2D is a graph showing an example of a transmission spectrum of theother of the two areas included in the plurality of areas in the filterarray shown in FIG. 2A;

FIG. 3A is a diagram for describing a relationship between the targetwavelength region and a plurality of wavelength regions includedtherein;

FIG. 3B is a diagram for describing a relationship between the targetwavelength region and the plurality of wavelength regions includedtherein;

FIG. 4A is a graph for describing characteristics of a transmissionspectrum in a certain area in the filter array;

FIG. 4B is graph showing a result of averaging the transmission spectrafor each of the wavelength regions shown in FIG. 4A;

FIG. 5 is a sectional view schematically showing a light detectingdevice in the exemplary embodiment;

FIG. 6 is a graph schematically showing an example of transmissionspectra of pixels;

FIG. 7 is a graph showing one example of a calculation result oftransmission spectra of a Fabry-Perot filter;

FIG. 8A includes graphs showing respective transmission spectra of ninetypes of multi-mode filter;

FIG. 8B includes graphs showing respective transmission spectra of ninetypes of single-mode filter;

FIG. 8C is a view showing original images and two examples ofreconstructed separate images;

FIG. 8D is a graph showing a calculation result of mean squared errorsbetween the original image and each reconstructed separate image;

FIG. 9 is a diagram schematically showing an example of two Fabry-Perotfilters whose thicknesses of intermediate layers are the closest in thefilter array;

FIG. 10 is a graph for describing the wavelengths of light detected bytwo pixels when light is orthogonally or obliquely incident on aFabry-Perot filter;

FIG. 11A is a graph showing incident angle dependency of a transmissionspectrum of a Fabry-Perot filter including an intermediate layer with arefractive index=1.5;

FIG. 11B is a graph showing incident angle dependency of a transmissionspectrum of a Fabry-Perot filter including an intermediate layer with arefractive index=2.35;

FIG. 12A is a view schematically showing a first modification of thelight detecting device shown in FIG. 5 ;

FIG. 12B is a view schematically showing a second modification of thelight detecting device shown in FIG. 5 ;

FIG. 12C is a view schematically showing a third modification of thelight detecting device shown in FIG. 5 ;

FIG. 12D is a view schematically showing a fourth modification of thelight detecting device shown in FIG. 5 ;

FIG. 12E is a view schematically showing a fifth modification of thelight detecting device shown in FIG. 5 ; and

FIG. 12F is a view schematically showing a sixth modification of thelight detecting device shown in FIG. 5 ;

FIG. 13 is a view schematically showing a light detecting device inanother exemplary embodiment;

FIG. 14A is a graph schematically showing an example of the transmissionspectra of filters A, B, and C and an example of the transmissionspectrum of a band-pass filter;

FIG. 14B is a diagram schematically showing an example of thetransmission spectrum of one filter constituted by a set of each offilters A, B, and C and a band-pass filter;

FIG. 15A is a diagram showing a modification of the example shown inFIG. 14A;

FIG. 15B is a diagram showing a modification of the example shown inFIG. 14B;

FIG. 16A is a diagram schematically showing a first modification of thelight detecting device shown in FIG. 13 ;

FIG. 16B is a diagram schematically showing a second modification of thelight detecting device shown in FIG. 13 ;

FIG. 16C is a diagram schematically showing a third modification of thelight detecting device shown in FIG. 13 ; and

FIG. 16D is a diagram schematically showing a fourth modification of thelight detecting device shown in FIG. 13 .

DETAILED DESCRIPTION

Before embodiments of the present disclosure are described, adescription will be given of knowledge underlying the presentdisclosure.

U.S. Patent Application Publication No. 2016/138975 discloses an imagingdevice that can acquire a high-resolution multi-wavelength image. In theimaging device, an optical element called an “encoding element” encodesan optical image from a target object to perform imaging. The encodingelement has a plurality of areas that is two-dimensionally arrayed. Thetransmission spectrum of each of at least two areas of the plurality ofareas has local maximum values of transmittance in respective wavelengthregions. The areas are disposed, for example, so as to respectivelycorrespond to pixels of an image sensor. In imaging using the encodingelement, data of each pixel includes information of a plurality ofwavelength regions. That is, image data that is generated is dataresulting from compression of wavelength information. Accordingly, it issufficient to hold two-dimensional data, thus making it possible toreduce the amount of data. For example, even when the capacity of arecording medium has a constraint, it is possible to obtain data oflong-term video.

The encoding element can be manufactured using various methods. Forexample, a method using organic material, such as dye or colorant, isconceivable. In this case, the areas in the encoding element are formedof light-absorbing materials having different light transmissioncharacteristics. In such a structure, the number of manufacturing stepsincreases according to the number of types of light-absorbing materialthat are disposed. Thus, it is not easy to fabricate the encodingelement using organic materials.

Meanwhile, U.S. Pat. Nos. 7,907,340, and 9,929,206, Japanese UnexaminedPatent Application Publication (Translation of PCT Application) No.2013-512445, and Japanese Unexamined Patent Application Publication(Translation of PCT Application) No. 2015-501432 disclose devices eachincluding a plurality of Fabry-Perot filters having transmission spectrathat are different from each other. Since Fabry-Perot filters allowtransmittances to be controlled with the shape of the structure thereof,they can be more easily fabricated than organic material when a largenumber of different filters are prepared. However, in any of theexamples disclosed in U.S. Pat. No. 7,907,340, and 9,929,206, JapaneseUnexamined Patent Application Publication (Translation of PCTApplication) No. 2013-512445, and Japanese Unexamined Patent ApplicationPublication (Translation of PCT Application) No. 2015-501432, data ofeach pixel includes only information of a single wavelength region.Thus, in order to acquire a plurality of pieces of color information, itis necessary to use a number of pixels that is equal to the number ofcolors. Consequently, the spatial resolution is sacrificed.

Based on the consideration above, the present inventors have conceivedan optical filter, a light detecting device, and a light detectingsystem recited in the following items.

(First Item)

An optical filter according to a first item includes a filter arrayincluding a plurality of filters two-dimensionally arrayed and aband-pass filter. The plurality of filters including a first filter anda second filter. A transmission spectrum of each of the first filter andthe second filter has local maximum values of transmittance at three ormore wavelengths included in a first wavelength region. The band-passfilter passes light in a second wavelength region including two or morewavelengths of the three or more wavelengths and not including one ormore wavelengths of the three or more wavelengths. The filter array andthe band-pass filter are disposed so that (a) the band-pass filter islocated on an optical path of transmitted light that passes through thefirst filter and the second filter or (b) the first filter and thesecond filter are located on an optical path of transmitted light thatpasses through the band-pass filter.

This optical filter makes it possible to reduce the amount of time for ahyperspectral camera to obtain multi-wavelength information.

(Second Item)

In the optical filter according to the first item, each of the firstfilter and the second filter may include a first reflective layer, asecond reflective layer, an intermediate layer between the firstreflective layer and the second reflective layer and may have aresonance structure having a plurality of resonant modes whose ordersare different from each other. At least one selected from the groupconsisting of a refractive index and a thickness of the intermediatelayer in the first filter may differ from the at least one selected fromthe group consisting of a refractive index and a thickness of theintermediate layer in the second filter. In other words, a refractiveindex of the intermediate layer in the first filter may be differentfrom a refractive index of the intermediate layer in the second filterand/or a thickness of the intermediate layer in the first filter may bedifferent from a thickness of the intermediate layer in the secondfilter. The plurality of resonant modes may include three resonantmodes. The three or more wavelengths may correspond to the threeresonant modes, respectively. In other words, each of the three or morewavelengths may correspond to a corresponding one of the three resonantmodes.

This optical filter provides an advantage that is the same as that ofthe optical filter according to the first item.

(Third Item)

In the optical filter according to the second item, each of theplurality of filters may have the resonance structure.

This optical filter provides an advantage that is the same as that ofthe optical filter according to the first or second item.

(Fourth Item)

In the optical filter according to one of the first to third items, thefilter array may further include a transparent layer for planarizing alevel difference between a surface of the first filter and a surface ofthe second filter.

In this optical filter, the transparent layer planarizes the leveldifference between the surface of the first filter and the surface ofthe second filter to thereby facilitate that another member is disposedon the transparent layer.

(Fifth Item)

In the optical filter according to one of the first to fourth items, atleast one of the plurality of filters may be transparent.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to fourth items.

(Sixth Item)

In the optical filter according to one of the first to fifth items, theplurality of filters may include a third filter that is different fromthe first filter and the second filter, and a transmission spectrum ofthe third filter may have one or more local maximum values oftransmittance.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to fifth items.

(Seventh Item)

In the optical filter according to one of the first to sixth items, thefilter array may be separated from the band-pass filter.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to sixth items.

(Eighth Item)

In the optical filter according to one of the first to sixth items, thefilter array may be in contact with the band-pass filter.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to sixth items.

(Ninth Item)

In the optical filter according to one of the first to eighth items, theband-pass filter may overlap the entire filter array when viewed in adirection orthogonal to the filter array.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to eighth items.

(Tenth Item)

In the optical filter according to one of the first to ninth items, theone or more wavelengths not included in the second wavelength region maybe located between a longest wavelength and a shortest wavelength of thetwo or more wavelengths included in the second wavelength region.

This optical filter provides an advantage that is the same as that ofthe optical filter according to one of the first to ninth items, evenwhen the second wavelength region includes a plurality of discretewavelength regions.

(11th Item)

A light detecting device according to an 11th item includes: an opticalfilter according to one of the first to tenth items; and an image sensorincluding a plurality of light detection elements, each havingsensitivity to light in the first wavelength region.

In this light detecting device, the image sensor can detect lightcomponents that are included in the first wavelength region and fromwhich light components other than the second wavelength region areeliminated by the optical filter.

(12th Item)

In the light detecting device according to the 11th item, the opticalfilter may be in contact with the image sensor.

This light detecting device provides an advantage that is the same asthat of the light detecting device according to the 11th item.

(13th Item)

In the light detecting device according to the 11th item, the opticalfilter may be separated from the image sensor.

This light detecting device provides an advantage that is the same asthat of the light detecting device according to the 11th item. The lightdetecting device according to the 11th item may further comprise opticalsystem including at least one lens. The optical system, the opticalfilter and the image sensor may be arranged in this order such thatlight enters from the optical system toward the image sensor.

(14th Item)

A light detecting system according to a 14th item includes: the lightdetecting device according to one of the 11th to 13th items; and asignal processing circuit. Based on a signal output from the imagesensor, the signal processing circuit generates and outputs image dataincluding information of a plurality of wavelengths included in thesecond wavelength region.

This light detecting system can generate and output image data includinginformation of a plurality of wavelengths included in the secondwavelength region.

(15th Item)

In the light detecting system according to the 14th item, the image datamay include data representing images spectrally separated for theplurality of wavelengths.

In this light detecting system, image data including data representingimages spectrally separated for the respective wavelengths.

(16th Item)

In the light detecting system according to the 15th item, the number ofpicture elements included in each of the images may be larger than M/N,where N represents the number of images, and M represents the number oflight detection elements.

This light detecting system makes it possible to suppress a reduction inthe resolution of each of the plurality of images.

(17th Item)

In the light detecting system according to the 16th item, the number ofpicture elements may be equal to the number of light detection elements.

This light detecting system can provide an advantage that is the same asthat of the 16th item.

In the present disclosure, all or a part of circuits, units, devices,members, or portions or all or a part of functional blocks in the blockdiagrams can be implemented by, for example, one or more electroniccircuits including a semiconductor device, a semiconductor integratedcircuit (IC), or a large-scale integration (LSI). The LSI or IC may beintegrated into one chip or may be constituted by combining a pluralityof chips. For example, functional blocks other than a storage elementmay be integrated into one chip. Although the name used here is an LSIor IC, it may also be called a system LSI, a very large scaleintegration (VLSI), or an ultra large scale integration (ULSI) dependingon the degree of integration. A field programmable gate array (FPGA)that can be programmed after manufacturing an LSI or a reconfigurablelogic device that allows reconfiguration of the connection relationshipor setup of circuit cells inside the LSI can also be used for the samepurpose.

In addition, functions or operations of all or a part of circuits,units, devices, members, or portions can be executed by softwareprocessing. In this case, the software is recorded on one or morenon-transitory recording media, such as a ROM, an optical disk, or ahard disk drive, and when the software is executed by a processingdevice (a processor), the processing device (the processor) andperipheral devices execute the functions specified by the software. Asystem or a device may include one or more non-transitory recordingmedia on which the software is recorded, a processing device (aprocessor), and necessary hardware devices, for example, an interface.

More specific embodiments of the present disclosure will be describedbelow with reference to the accompanying drawings. However, an overlydetailed description may be omitted. For example, a detailed descriptionof already well-known things and a redundant description ofsubstantially the same configurations may be omitted herein. This is toavoid the following description becoming overly redundant and tofacilitate understanding of those skilled in the art. The inventorsprovide the accompanying drawings and the following description in orderfor those skilled in the art to fully understand the present disclosure,and these drawings and description are not intended to limit the subjectmatter as recited in the claims. In the following description, the sameor similar constituent elements are denoted by the same referencenumerals.

Embodiments

<Light Detecting System>

First, a light detecting system in the present embodiment will bedescribed.

FIG. 1 is a diagram schematically showing a light detecting system 400in an exemplary embodiment. The light detecting system 400 includes anoptical system 40, a filter array 100C, an image sensor 60, and a signalprocessing circuit 200. The filter array 100C has functions that aresimilar to those of the “encoding element” disclosed in U.S. PatentApplication Publication No. 2016/138975. Thus, the filter array 100C canalso be referred to as an “encoding element”. The optical system 40 andthe filter array 100C are disposed on an optical path of light that isincident from a target object 70.

The filter array 100C includes a plurality of light-transmissive areasarrayed in a row or column. The filter array 100C is an optical elementin which the light transmission spectrum, that is, the wavelengthdependency of the light transmittance, differs depending on the area.The filter array 100C modulates the intensity of incident light andpasses the modulated incident light. The filter array 100C may bedisposed in the vicinity of or directly above the image sensor 60. The“vicinity” as used herein means being in close proximity to a degreethat an optical image from the optical system 40 is formed on a plane ofthe filter array 100C in a clear state to some degree. The “directlyabove” means that both are in close proximity to each other to a degreethat almost no gap occurs therebetween. The filter array 100C and theimage sensor 60 may be integrated together. A device including thefilter array 100C and the image sensor 60 is referred to as a “lightdetecting device 300”.

The optical system 40 includes at least one lens. Although the opticalsystem 40 is shown as one lens in FIG. 1 , the optical system 40 may beconstituted by a combination of a plurality of lens. The optical system40 forms an image on an imaging plane of the image sensor 60 via thefilter array 100C.

Based on an image 120 acquired by the image sensor 60, the signalprocessing circuit 200 reconstructs a plurality of separate images 220including multi-wavelength information. Details of the plurality ofseparate images 220 and a processing method for image signals in thesignal processing circuit 200 are described later. The signal processingcircuit 200 may be incorporated into the light detecting device 300 ormay be a constituent element of a signal processing device electricallyconnected to the light detecting device 300 by wire or wirelessly.

<Filter Array>

The filter array 100C in the present embodiment will be described below.The filter array 100C is used in a spectroscopy system that generatesimages for respective wavelength regions included in a wavelength regionto be imaged. Herein, the wavelength region to be imaged may be referredto as a “target wavelength region”. The filter array 100C is disposed onan optical path of light that is incident from a target object,modulates the intensity of the incident light for respectivewavelengths, and outputs the modulated light. This process involving thefilter array 100C, that is, the encoding element, is herein referred toas “encoding”.

FIG. 2A is a view schematically showing an example of the filter array100C. The filter array 100C has a plurality of areas that istwo-dimensionally arrayed. The areas may herein be referred to as“cells”. A filter having a transmission spectrum that is individuallyset is disposed in each area. The transmission spectrum is representedby a function T(X), where X is the wavelength of incident light. Thetransmission spectrum T(X) can take a value that is greater than orequal to 0 and is less than or equal to 1. Details of the configurationof the filters are described below.

In the example shown in FIG. 2A, the filter array 100C has 48rectangular areas arrayed in six rows by eight columns. This is merelyexemplary, and a larger number of areas than that number can be providedin actual applications. The number of areas can be, for example,approximately the same as the number of pixels in a typicalphotodetector, such as an image sensor. The number of pixels is, forexample, a few hundred thousand to tens of millions. In one example, thefilter array 100C may be disposed directly above a photodetector, andeach area may be disposed so as to correspond to one pixel in thephotodetector. Each area faces, for example, one pixel in thephotodetector.

FIG. 2B is a view showing one example of spatial distributions oftransmittances of light in respective wavelength regions W1, W2, . . . ,and Wi included in a target wavelength region. In the example shown inFIG. 2B, differences in darkness/lightness in the areas representdifferences in the transmittances. The lighter the area is, the higherthe transmittance is, and the darker the area is, the lower thetransmittance is. As shown in FIG. 2B, the spatial distribution of thelight transmittances differs depending on the wavelength region.

FIG. 2C and FIG. 2D are graphs showing examples of transmission spectrain area A1 and area A2, respectively, included in the areas in thefilter array 100C shown in FIG. 2A. The transmission spectrum in area A1and the transmission spectrum in area A2 differ from each other. Thus,the transmission spectrum of the filter array 100C differs depending onthe area. However, the transmission spectra in all the areas do notnecessarily have to differ from each other. In the filter array 100C,the transmission spectra of at least some of the areas are differentfrom each other. The at least some of the areas are two or more areas.That is, the filter array 100C includes two or more filters whosetransmission spectra are different from each other. In one example, thenumber of patterns of the transmission spectra of the areas included inthe filter array 100C can be greater than or equal to the number i ofwavelength regions included in the target wavelength region. The filterarray 100C may be designed such that the transmission spectra of morethan half of the areas are different.

FIGS. 3A and 3B are diagrams for describing a relationship between atarget wavelength region W and the wavelength regions W1, W2, . . . ,and Wi included therein. The target wavelength region W can be set tovarious ranges, depending on the application. The target wavelengthregion W can be, for example, a visible-light wavelength region of about400 nm to about 700 nm, a near-infrared wavelength region of about 700nm to about 2500 nm, and a near-ultraviolet wavelength region of about10 nm to about 400 nm, or radio wave ranges of mid infrared, farinfrared, terahertz waves, millimeter waves, or the like. Thus, thewavelength region that is used is not limited to a visible light region.Herein, not only visible light but also non-visible light, such asnear-ultraviolet, near-infrared, and radio waves, is referred to as“light”, for the sake of convenience”.

In the example shown in FIG. 3A, i wavelength regions obtained byequally dividing the target wavelength region W are referred to as“wavelength regions W1, W2, . . . , and Wi”, where i is an arbitraryinteger greater than or equal to 4. However, the present disclosure isnot limited to such an example. The plurality of wavelength regionsincluded in the target wavelength region W may be arbitrarily set. Forexample, the bandwidths may be made unequal depending on the wavelengthregion. A gap may be present between the adjacent wavelength regions. Inthe example shown in FIG. 3B, the bandwidth differs depending on thewavelength region, and a gap exists between two adjacent wavelengthregions. In such a manner, it is sufficient as long as the wavelengthregions be different from each other, and how they are determined isarbitrary. The number i of divided wavelengths may be less than or equalto 3.

FIG. 4A is a graph for describing characteristics of a transmissionspectrum in a certain area in the filter array 100C. In the exampleshown in FIG. 4A, the transmission spectrum has a plurality of localmaximum values P1 to P5 and a plurality of local minimum values withrespect to the wavelengths in the target wavelength region W. In theexample shown in FIG. 4A, normalization is performed so that the maximumvalue of the light transmittance in the target wavelength region W is 1,and the minimum value thereof is 0. In the example shown in FIG. 4A, thetransmission spectrum has the local maximum values in wavelengthregions, such as wavelength regions W2 and Wi−1. Thus, in the presentembodiment, the transmission spectrum in each area has the local maximumvalues at at least two wavelengths of the wavelength regions W1 to Wi.As can be seen from FIG. 4A, the local maximum values P1, P3, P4, and P5are greater than or equal to 0.5.

As described above, the light transmittance in each area differsdepending on the wavelength. Thus, the filter array 100C passes a largeamount of components in one wavelength region of incident light and doesnot pass much components in other wavelength regions of the incidentlight. For example, the transmittances for light in k wavelength regionsof the i wavelength regions can be larger than 0.5, and thetransmittances for light in the remaining i-k wavelength regions can besmaller than 0.5, where k is an integer that satisfies 2≤k<i. Ifincident light is white light equally including wavelength components ofall visible light, the filter array 100C modulates, for each area, theincident light into light having a plurality of discrete intensity peakswith respect to the wavelength, superimposes the light having multiwavelengths, and outputs the light.

FIG. 4B is a graph showing one example of a result of averaging thetransmittances shown in FIG. 4A for each of wavelength regions W1, W2, .. . , and Wi. The averaged transmittance is obtained by integrating thetransmission spectrum T(λ) for each wavelength region and dividing theresult thereof by the bandwidth of the wavelength region. The value ofthe transmittance averaged for each wavelength region in such a mannerwill herein be referred to as a “transmittance” in the wavelengthregion. In this example, in three wavelength regions in which the localmaximum values P1, P3, and P5 are reached, the transmittances areprominently high. In particular, in two wavelength regions where thelocal maximum values P3 and P5 are reached, the transmittances exceed0.8.

The resolution in a wavelength direction of the transmission spectrum ineach area can be set to approximately the bandwidth of a desiredwavelength region. In other words, in a wavelength range including onelocal maximum value on a transmission spectrum curve, the width of arange that takes values that are greater than or equal to the averagevalue of the local maximum value and a local minimum value that is themost adjacent to the local maximum value can be set to approximately thebandwidth of a desired wavelength region. In this case, when thetransmission spectrum is resolved into frequency components, forexample, by a Fourier transform, the values of frequency componentscorresponding to the wavelength region increase relatively.

The filter array 100C is typically divided into a plurality of cellssectioned into a lattice form, as shown in FIG. 2A. These cells havetransmission spectra that are different from each other. The wavelengthdistribution and the spatial distribution of the light transmittances ofthe areas in the filter array 100C can be, for example, randomdistributions or pseudo-random distributions.

The concepts of the random distribution and the pseudo-randomdistribution are as follows. First, each area in the filter array 100Ccan be regarded as, for example, a vector element having a value of 0 to1 according to the light transmittance. In this case, when thetransmittance is 0, the value of the vector element is 0, and when thetransmittance is 1, the value of the vector element is 1. In otherwords, a collection of areas that are arranged in one line in a rowdirection or a column direction can be regarded as a multidimensionalvector having values of 0 to 1. Accordingly, the filter array 100C canbe said to include a plurality of multidimensional vectors in the columndirection or the row direction. In this case, the random distributionmeans that two arbitrary multidimensional vectors are independent fromeach other, that is, are not parallel to each other. Also, thepseudo-random distribution means that a non-independent configuration isincluded between some multidimensional vectors. Accordingly, in therandom distribution and the pseudo-random distribution, a vector whoseelements are the values of the transmittances of light in a firstwavelength region in the areas belonging to a collection of the areasarranged in one row or column included in the plurality of areas and avector whose elements are the values of the transmittances of light inthe first wavelength region in the areas belonging to a collection ofthe areas arranged in another row or column are independent from eachother. In a second wavelength region that is different from the firstwavelength region, similarly, a vector whose elements are the values ofthe transmittances of light in the second wavelength region in the areasbelonging to a collection of the areas arranged in one row or columnincluded in the plurality of areas and a vector whose elements are thevalues of the transmittances of light in the second wavelength region inthe areas belonging to a collection of the areas arranged in another rowor column are intendent from each other.

When the filter array 100C is disposed in the vicinity of or directlyabove the image sensor 60, a cell pitch, which is the interval betweenthe areas in the filter array 100C, may be made to generally match thepixel pitch of the image sensor 60. With this arrangement, theresolution of an encoded optical image emitted from the filter array100C generally matches the resolution of the pixels. When light thatpasses through each cell is adapted to be incident on only onecorresponding pixel, it is possible to easily perform an arithmeticoperation described below. When the filter array 100C is disposed awayfrom the image sensor 60, the cell pitch may be reduced according to thedistance therebetween.

In the example shown in FIGS. 2A to 2D, a grayscale transmittancedistribution in which the transmittance of each area can take a valuethat is greater than or equal to 0 and is less than or equal to 1 isenvisaged. However, the transmittance distribution does not necessarilyhave to be made to be a grayscale transmittance distribution. Forexample, a binary-scale transmittance distribution in which thetransmittance of each area can take a value of either generally 0 orgenerally 1 may be employed. In the binary-scale transmittancedistribution, each area passes a majority of light in at least twowavelength regions of wavelength regions included in the targetwavelength region and does not pass a majority of light in the remainingwavelength regions. The “majority” herein refers to about 80% or more.

Some of all the cells, for example, half of the cells, may be replacedwith transparent areas. Such transparent areas pass light in all thewavelength regions W1 to Wi included in the target wavelength region atapproximately the same high transmittance. The high transmittance is,for example, greater than or equal to 0.8. In such a configuration, thetransparent areas can be disposed, for example, in a checkered pattern.That is, in two array directions of the areas in the filter array 100C,the area in which the light transmittance differs depending on thewavelength and the transparent area can be alternately arrayed. In theexample shown in FIG. 2A, the two array directions are a horizontaldirection and a vertical direction.

<Signal Processing Circuit>

Next, a description will be given of a method in which the signalprocessing circuit 200 shown in FIG. 1 reconstructs multi-wavelengthseparate images 220 based on the image 120 and spatial distributioncharacteristics of transmittances for respective wavelengths in thefilter array 100C. The “multi wavelengths” as used herein means a largernumber of wavelength regions than the wavelength regions in three colorsof RGB obtained by, for example, an ordinary color camera. The number ofwavelength regions can be, for example, about 4 to 100. The number ofwavelength regions may be referred to as the “number of spectral bands”.The number of spectral bands may exceed 100, depending on theapplication.

Data to be acquired are separate images 220, and the data arerepresented as f. When the number of spectral bands is represented as w,f is data obtained by integrating pieces of image data f₁, f₂, . . . ,and f_(w) in the individual bands. When the number of picture elementsin an x-direction of the image data to be acquired is represented as n,and the number of picture elements in a y-direction thereof isrepresented as m, each of the pieces of image data f₁, f₂, . . . , andf_(w) is a collection of pieces of two-dimensional data for n×m pictureelements. Accordingly, the data f is three-dimensional data in which thenumber of elements is n×m×w. Meanwhile, the number of elements in data gof the image 120 acquired through encoding and multiplexing performed bythe filter array 100C is n×m. The data g in the present embodiment canbe given by expression (1):

$\begin{matrix}{g = {{Hf} = {H\begin{bmatrix}f_{1} \\f_{2} \\\vdots \\f_{w}\end{bmatrix}}}} & (1)\end{matrix}$

In this case, f₁, f₂, . . . f_(w) are each a piece of data having n×melements. Thus, the vector on the right-hand side is, strictly speaking,a one-dimensional vector for n×m×w rows by one column. The vector g isconverted into and expressed by a one-dimensional vector for n×m rows byone column, and the one-dimensional vector is calculated. A matrix Hrepresents conversion for encoding and intensity-modulating thecomponents f₁, f₂, . . . , and f_(w) of the vector f with pieces ofencoding information that are different for each wavelength region andadding resulting components. Thus, H is a matrix with n×m rows by n×m×wcolumns.

When the vector g and the matrix H are given, it seems that f can bedetermined by solving an inverse problem of expression (1). However,since the number n×m×w of elements in the data f to be determined islarger than the number n×m of elements in the acquired data g, thisproblem is an ill-posed problem and cannot be directly solved. Hence,the signal processing circuit 200 in the present embodiment utilizesredundancy of images included in the data f to determine the solution byusing a scheme for compressive sensing. Specifically, the data f to bedetermined is estimated by solving expression (2):

$\begin{matrix}{f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {\tau{\Phi(f)}}} \right\}}} & (2)\end{matrix}$

In this case, f′ represents data of estimated f. The first term in thebraces in the above equation represents the amount of deviation betweenan estimation result Hf and the acquired data g, that is, the so-calledresidual term. Although the sum of squares is used as the residual term,an absolute value, the square root of the sum of squares, or the likemay be used as the residual term. The second term in the braces is aregularization term or a stabilization term described below. Expression(2) means determining f that minimizes the sum of the first term and thesecond term. The signal processing circuit 200 can calculate the finalsolution f′ by converging solutions through a recursive iterationarithmetic operation.

The first term in the braces in expression (2) means an arithmeticoperation for determining the sum of squares of differences between theacquired data g and Hf obtained by system-converting fin an estimationprocess by using the matrix H. Φ(f) in the second term is a constraintcondition in regularization of f and is a function that reflects sparseinformation of the estimated data. OM serves to provide an effect ofsmoothing or stabilizing the estimated data. The regularization term canbe represented by, for example, a discrete cosine transform (DCT), awavelet transform, a Fourier transform, total variation (TV), or thelike of f. For example, when total variation is used, it is possible toacquire stable estimated data with reduced influences of noise in theobserved data Φ(f). The sparseness of the target object 70 in the spaceof each regularization term differs depending on the texture of thetarget object 70. A regularization term with which the texture of thetarget object 70 becomes sparser in the space of the regularization termmay be selected. Alternatively, the arithmetic operation may include aplurality of regularization terms. As a weighting factor τ increases,the amount of reduction in redundant data increases, and the rate ofcompression increases. In expression (2), τ is a weighting factor. Asthe weighting factor τ decreases, the convergence to the solutionbecomes weak. The weighting factor τ is set to an appropriate value atwhich f converges to some degree and no over-compression occurs.

Although an arithmetic operation example using the compressive sensingindicated in expression (2) has been described above, another method maybe used to determine the solution. For example, another statisticalmethod, such as a maximum likelihood estimation method or a Bayesianestimation method, can be used. Also, the number of separate images 220is arbitrary, and each wavelength region may also be arbitrarily set.Details of the method for the reconstruction are disclosed in U.S.Patent Application Publication No. 2016/138975. The entire contentsdisclosed in U.S. Patent Application Publication No. 2016/138975 arehereby incorporated by reference.

<Filter Array Including Fabry-Perot Filters>

Next, a description will be given of an example of a more specificstructure of the filter array 100C.

FIG. 5 is a sectional view schematically showing the light detectingdevice 300 in the exemplary embodiment. The light detecting device 300includes the filter array 100C and the image sensor 60.

The filter array 100C includes a plurality of filters 100 that istwo-dimensionally arrayed. The filters 100 are arrayed in a row and acolumn, for example, as shown in FIG. 2A. FIG. 5 schematically shows across-sectional structure of one of the rows shown in FIG. 2A. Each ofthe filters 100 has a resonance structure. The resonance structure meansa structure in which light with a certain wavelength forms a standingwave and exists stably. The state of the light may be referred to as a“resonant mode”. The resonance structure shown in FIG. 5 includes afirst reflective layer 28 a, a second reflective layer 28 b, and anintermediate layer 26 between the first reflective layer 28 a and thesecond reflective layer 28 b. The first reflective layer 28 a and/or thesecond reflective layer 28 b can be formed of a dielectric multilayerfilm or a metal thin film. The intermediate layer 26 can be formed of adielectric or semiconductor that is transparent in a specific wavelengthregion. The intermediate layer 26 can be formed from, for example, atleast one selected from the group consisting of silicon (Si), siliconnitride (Si₃N₄), titanium dioxide (TiO₂), niobium pentoxide (Nb₂O₅), andtantalum pentoxide (Ta₂O₅). The refractive index and/or the thickness ofthe intermediate layer 26 in the filters 100 differ depending on thefilter. The transmission spectrum of each of the filters 100 has localmaximum values of transmittance at a plurality of wavelengths. Thewavelengths respectively correspond to resonant modes whose orders aredifferent from each other in the above-described resonance structure. Inthe present embodiment, all the filters 100 in the filter array 100Cinclude the above-described resonance structures. The filter array 100Cmay include filters that do not have the above-described resonancestructures. For example, the filter array 100C may include filters, suchas transparent filters or neutral density (ND) filters, that do not havewavelength dependency of the light transmittance. In the presentdisclosure, two or more of the filters 100 each has the above-describedresonance structure.

The image sensor 60 includes a plurality of light detection elements 60a. Each of the light detection elements 60 a is disposed to face one ofthe plurality of filters 100. Each of the light detection elements 60 ahas sensitivity to light in a specific wavelength region. This specificwavelength region corresponds to the above-described target wavelengthregion W. In the present disclosure, “having sensitivity to light in acertain wavelength region” refers to having substantial sensitivityneeded to detect light in the wavelength region. For example, it meansthat an external quantum efficiency in the wavelength region is 1% ormore. An external quantum efficiency of the light detection elements 60a may be 10% or more. The external quantum efficiency of the lightdetection elements 60 a may be 20% or more. The wavelengths at which thelight transmittance of each filter 100 takes the local maximum valuesare all included in the target wavelength region W. The light detectionelements 60 a may be referred to as “pixels” in the description below.

Besides the example shown in FIG. 5 , the filter array 100C and theimage sensor 60 may be separated from each other. Even in such a case,each of the light detection elements 60 a is disposed at a positionwhere it receives light that has passed through one of the filters.Constituent elements may be disposed so that light that have passedthrough the filters are incident on the light detection elements 60 a,respectively, through a mirror. In this case, each of the lightdetection elements 60 a is not disposed directly below one of thefilters.

Herein, the filters 100 having the above-described resonance structuresmay be referred to as “Fabry-Perot filters”. Herein, the portion of atransmission spectrum having a local maximum value may be referred to asa “peak”, and a wavelength at which the transmission spectrum has alocal maximum value may be referred to as a “peak wavelength”.

Next, a description will be given of the transmission spectra of thefilters 100, which are Fabry-Perot filters.

In this case, a peak wavelength λ_(m) of the transmission spectrum ofthe filter 100 is given by:

$\begin{matrix}{\lambda_{m} = {\frac{2nL}{m}\sqrt{1 - \left( \frac{\sin\;\theta_{i}}{n} \right)^{2}}}} & (3)\end{matrix}$where, in each filter 100, L represents the thickness of theintermediate layer 26, n represents the refractive index, θ_(i)represents the incident angle of light that is incident on the filter100, and m represents the mode number of the resonant mode and is aninteger greater than or equal to 1.

A shortest wavelength in the target wavelength region W is representedby λ_(i), and a longest wavelength therein is represented by λ_(e).Herein, the filter 100 with which the number of m's that satisfyλ_(i)≤λ_(m)≤λ_(e) is one is referred to as a “single-mode filter”. Thefilter 100 with which the number of m's that satisfy λ_(i)≤λ_(m)≤λ_(e)is two or more is referred to as a “multi-mode filter”. An example whenthe shortest wavelength in the target wavelength region W is given byλ_(i)=400 nm, and the longest wavelength therein is given by λ_(e)=700nm will be described below.

For example, in the filter 100 with the thickness L=300 nm, therefractive index n=1.0, and the orthogonal incidence θ_(i)=0°, the peakwavelength for m=1 is λ₁=600 nm, and the peak wavelength for m≥2 isλ_(m≥2)≤2300 nm. Accordingly, this filter 100 is a single-mode filterwith which one peak wavelength is included in the target wavelengthregion W.

On the other hand, when the thickness L is increased to a thicknesslarger than 300 nm, a plurality of peak wavelengths is included in thetarget wavelength region W. For example, in the filter 100 with thethickness L=3000 nm, n=1.0, and the orthogonal incidence θ_(i)=0°, thepeak wavelength for 1≤m≤8 is λ_(i≤m≤8)≤750 nm, the peak wavelength for9≤m≤15 is 400 nm≤λ_(9≤m≤15)≤700 nm, and the peak wavelength for m≥16 isλ_(m≥16)≤375 nm. Accordingly, this filter 100 is a multi-mode filterwith which seven peak wavelengths are included in the target wavelengthregion W.

Appropriately designing the thickness of the intermediate layer 26 inthe filter 100, as described above, makes it possible to realize amulti-mode filter. The refractive index of the intermediate layer 26,instead of the thickness of the intermediate layer 26, in the filter 100may be appropriately designed. Alternatively, both the thickness and therefractive index of the intermediate layer 26 in the filter 100 may beappropriately designed.

FIG. 6 is a graph schematically showing an example of transmissionspectra at pixels when multi-mode filters whose transmission spectra aredifferent from each other are respectively disposed above the pixels,which are the light detection elements 60 a. FIG. 6 illustratestransmission spectra at pixels A, B, and C. The multi-mode filters aredesigned so that the peak wavelengths are slightly different from onepixel to another. Such a design can be realized by slightly varying thethickness L and/or the refractive index n in expression (3). In thiscase, in each pixel, a plurality of peaks appears in the targetwavelength region W. The mode numbers of the respective peaks are thesame among the pixels. The mode numbers of the respective peaks shown inFIG. 6 are m, m+1, and m+2. The light detecting device 300 in thepresent embodiment can simultaneously detect light with a plurality ofpeak wavelengths that differ from one pixel to another.

FIG. 7 is a graph showing one example of a calculation result oftransmission spectra of the filters 100. In this example, each of thefirst reflective layer 28 a and the second reflective layer 28 b in eachfilter 100 is formed of a dielectric multilayer film in which a TiO₂layer and a silicon dioxide (SiO₂) layer are alternately stacked. Theintermediate layer 26 in the filter 100 is formed of a TiO₂ layer. Inthe example shown in FIG. 7 , the thickness of the intermediate layer 26corresponding to the transmission spectrum denoted by a solid line isdifferent from the thickness of the intermediate layer 26 correspondingto the transmission spectrum denoted by a dotted line. DiffractMOD basedon Rigorous Coupled-Wave Analysis (RCWA) of Synopsys Inc. (formerly,RSoft Inc.) was used for calculating the transmission spectra. As shownin FIG. 7 , the plurality of peak wavelengths in the target wavelengthregion W differs depending on the pixel. As described above, changingthe thickness of the intermediate layer 26 in the multi-mode filter 100for each pixel allows the light detecting device 300 in the presentembodiment to simultaneously detect light with a plurality of peakwavelengths that differ for each pixel.

Next, a plurality of separate images 220 reconstructed by a plurality ofmulti-mode filters will be described in comparison with a plurality ofseparate images 220 reconstructed by a plurality of single-mode filters.

FIG. 8A has graphs showing respective transmission spectra of nine typesof multi-mode filter. FIG. 8B has graphs showing respective transmissionspectra of nine types of single-mode filter. In the example shown inFIG. 8A, the transmission spectrum of each multi-mode filter exhibitseight or nine peaks in the target wavelength region W. In the exampleshown in FIG. 8B, the transmission spectrum of each single-mode filterexhibits one peak in the target wavelength region W. The filter array100C includes, for example, one million filters 100 that aretwo-dimensionally arrayed. The million filters 100 randomly include ninetypes of multi-mode filter shown in FIG. 8A. Alternatively, the millionfilters 100 randomly includes nine types of single-mode filter shown inFIG. 8B. Because of the random arrangement, adjacent filters may be ofthe same type of filter. Such cases, however, are thought to be rare.Accordingly, a major problem does not occur.

FIG. 8C is a view showing original images and two examples of aplurality of reconstructed separate images 220. The upper stage in FIG.8C shows original images. The middle stage in FIG. 8C shows an exampleof a plurality of separate images 220 reconstructed via the nine typesof multi-mode filter shown in FIG. 8A. The lower stage in FIG. 8C showsan example of a plurality of separate images 220 reconstructed via thenine types of single-mode filter shown in FIG. 8B. Thirty images in eachof the upper stage, the middle stage, and the lower stage wererespectively acquired by detecting light in 30 wavelength regions. The30 wavelength regions were obtained by equally dividing the targetwavelength region W from 400 nm to 700 nm into 30 regions in incrementsof 10 nm. For example, the first, second, and third images from the topleft in each of the upper stage, the middle stage, and the lower stagewere obtained by respectively detecting light in a wavelength regionfrom 400 nm to 410 nm, light in a wavelength region from 410 nm to 420nm, and light in a wavelength region from 420 nm to 430 nm,respectively. In the example shown in FIG. 8C, the images in the lowerstage are darker than the images in the middle stage. This is thought tobe because the amount of light that passed through the single-modefilters is smaller than the amount of light that passed through themulti-mode filters.

FIG. 8D is a graph showing a calculation result of mean squared errors(MSEs) between the original image and each reconstructed separate image220. The mean squared errors are calculated using expression (4):

$\begin{matrix}{{M\; S\; E} = {\frac{1}{N \cdot M}{\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{M}\left( {I_{i,j}^{\prime} - I_{i,j}} \right)^{2}}}}} & (4)\end{matrix}$

In this case, N and M are the number of picture elements in a verticaldirection and the number of picture elements in a lateral direction,respectively. I_(i,j) is a picture element value of the original imageat the picture element at position (i,j). I′_(i,j) is a picture elementvalue in each reconstructed separate image 220 at the picture element atposition (i,j).

As shown in FIG. 8D, the MSE between the original image and eachseparate image 220 reconstructed via the multi-mode filters issufficiently smaller than the MSE between the original image and eachseparate image 220 reconstructed via the single-mode filters.Accordingly, in the light detecting device 300 in the presentembodiment, the original images can be accurately reproduced with theplurality of separate images 220 reconstructed via the multi-modefilters. The original images cannot be accurately reproduced with theplurality of separate images 220 reconstructed via the single-modefilters.

As described above, in the light detecting system 400 in the presentembodiment, the signal processing circuit 200 shown in FIG. 1 generatesimage data including information of a plurality of wavelengths, based onsignals output from the plurality of pixels 60 a. The image dataincludes data representing the separate images 220 spectrally separatedfor the respective wavelengths.

The number of picture elements in each of the separate images 220 islarger than M/N, where N presents the number of separate images 220, andM represents the number of pixels. In the example shown in the middlestage in FIG. 8C, the number of picture elements is equal to M. Thus,even when the images are spectrally separated for the respectivewavelengths, the light detecting system 400 in the present embodimentcan suppress a reduction in the resolution of each of the plurality ofseparate images 220.

Next, a description will be given of an influence that the refractiveindices of the intermediate layers 26 in the filters 100 have on thewavelength resolution of the light detecting device 300.

As described above, the filters 100 in the filter array 100C include theintermediate layers 26 having thicknesses that are different from eachother, as shown in FIG. 5 . FIG. 9 is a diagram schematically showing anexample of two filters 100 whose thicknesses of the intermediate layers26 are the closest in the filter array 100C. In the example shown inFIG. 9 , the filter 100 in which the thickness of the intermediate layer26 is L and the filter 100 in which the thickness of the intermediatelayer 26 is L+ΔL are disposed adjacent to each other. The pixel forwhich the filter 100 in which the thickness of the intermediate layer 26is L is disposed and the pixel for which the filter 100 in which thethickness of the intermediate layer 26 is L+ΔL is disposed are referredto as “pixel A” and “pixel B”, respectively. In the filter array 100C,even when two filters 100 shown in FIG. 9 are disposed away from eachother, the following discussion holds true.

In the example shown in FIG. 9 , the light detecting device 300 includesthe optical system 40. When an optical image from a target object passesthrough the center of the lens included in the optical system 40, theoptical image is orthogonally incident on the filter 100. On the otherhand, when an optical image from a target object passes through aportion other than the center of the lens included in the optical system40, the optical image is obliquely incident on the filter 100 at afinite incident angle. The finite incident angle is determined by anumerical aperture NA of the optical system 40. That is, the minimumvalue of the incident angle of light that is incident on the filterarray 100C from the optical system 40 is 0°, and the maximum value ofthe incident angle is sin⁻¹(NA)°. Accordingly, in the transmissionspectrum of the filter 100, in accordance with expression (3), the peakwavelength shifts toward the short wavelength side as the incident angleθ_(i) increases.

FIG. 10 has graphs for describing the wavelengths of light detected bypixels A and B when light is orthogonally or obliquely incident on thefilter 100. The upper stage in FIG. 10 shows peaks of light detected bypixels A and B in the case of orthogonal incidence. The middle stage inFIG. 10 shows peaks of light detected by pixels A and B in the case ofoblique incidence. The lower stage in FIG. 10 shows peaks of lightdetected by pixel B in the case of oblique incidence. In the descriptionbelow, the peak wavelength of the mode number m for light detected bypixel A is referred to as a “peak wavelength of pixel A”, and the peakwavelength of the mode number m for light detected by pixel B isreferred to as a “peak wavelength of pixel B”. In the orthogonalincidence, the peak wavelength of pixel A is λ_(A)=2nL/m, and the peakwavelength of pixel B is λ_(B)=2n(L+ΔL)/m. The difference ΔL between thethicknesses of the intermediate layers 26 in the two filters 100 shownin FIG. 9 is the smallest of combinations of two arbitrary filters 100in the filter array 100C. Accordingly, an interval Δλ_(ΔL)=2 nΔL/m ofthe peak wavelengths of pixels A and B is the smallest of thecombinations of two arbitrary pixels in the image sensor 60. Theinterval of the peak wavelengths of pixels A and B corresponds to thewavelength resolution of the light detecting device 300.

On the other hand, in the oblique incidence, the peak wavelength shiftstoward the short wavelength side. An amount Δλ_(θi) of shift of the peakwavelength in the oblique incidence is given by expression (5):

$\begin{matrix}{{\Delta\lambda_{\theta_{i}}} = {\frac{2nL}{m}\left\lbrack {1 - \sqrt{1 - \left( \frac{\sin\;\theta_{i}}{n} \right)^{2}}} \right\rbrack}} & (5)\end{matrix}$

Thus, for Δλ_(θi)≥Δλ_(ΔL), there is a possibility that the peakwavelength of pixel B in the oblique incidence matches the peakwavelength of pixel A in the orthogonal incidence. In the example shownin FIG. 9 , both light of the orthogonal incidence and light of theoblique incidence are simultaneously detected by pixel B. Accordingly,for Δλ_(θi)≥Δλ_(ΔL), light that should be detected by pixel A is falselydetected by pixel B.

From the above discussion, it follows that a condition that the falsedetection does not occur is Δλ_(θi)<Δλ_(ΔL). When Δλ_(θi)<Δλ_(ΔL) ismodified, expression (6) below is obtained.

$\begin{matrix}{{\Delta L} > {L\left\lbrack {1 - \sqrt{1 - \left( \frac{\sin\theta_{i}}{n} \right)^{2}}} \right\rbrack}} & (6)\end{matrix}$

In addition, the condition that the false detection does not occur maybe set to Δλ_(θi)<Δλ_(ΔL)/2. In the following description, thewavelength region in which light with a peak wavelength at the modenumber m is detected by pixel A is referred to as “wavelength region A”,and the wavelength region in which light with a peak wavelength at themode number m is detected by pixel B is referred to as “wavelengthregion B”. When the upper limit of wavelength region A and the lowerlimit of wavelength region B are both set to(λ_(A)+λ_(B))/2=Δλ_(A)+Δλ_(ΔL)/2, the peak wavelength of image B doesnot enter wavelength region A even for oblique incidence, because ofΔλ_(θi)<Δλ_(ΔL)/2. This allows the signal processing circuit 200 toprocess a peak wavelength in wavelength region A as the peak wavelengthof pixel A and to process a peak wavelength in wavelength region B asthe peak wavelength of pixel B, regardless of the incident angle θ_(i).As a result, occurrence of the false detection can be more reduced thanin the case of expression (6). When Δλ_(θi)<Δλ_(ΔL)/2 is modified,expression (7) below is obtained.

$\begin{matrix}{{\Delta L} > {2{L\left\lbrack {1 - \sqrt{1 - \left( \frac{\sin\theta_{i}}{n} \right)^{2}}} \right\rbrack}}} & (7)\end{matrix}$

According to expressions (6) and (7), increasing the refractive index nof the intermediate layer 26 makes it possible to reduce the influencesof the incident angle θ_(i). As a result, it is possible to improve thewavelength resolution of the light detecting device 300.

FIG. 11A and FIG. 11B are graphs showing incident angle dependency ofthe transmission spectra of the filter 100 including the intermediatelayer 26 with the refractive index n=1.5 and the filter 100 includingthe intermediate layer 26 with the refractive index n=2.35,respectively. In the example shown in FIG. 11A, when the incident anglee changes from 0° to 30°, the peak wavelength shifts by 26.1 nm. In theexample shown in FIG. 11B, when the incident angle changes from 0° to30°, the peak wavelength shifts by 17.1 nm. That is, increasing therefractive index of the intermediate layer 26 makes it possible toreduce the amount of shift in the peak wavelength, the shift beingcaused by a change in the incident angle θ_(i). Accordingly,appropriately designing the refractive index of the intermediate layer26 in accordance with expressions (6) and (7) makes it possible toimprove the wavelength resolution of the light detecting device 300.

A difference between the light detecting device 300 in the presentembodiment and the device disclosed in U.S. Pat. No. 9,929,206 will bedescribed next.

U.S. Pat. No. 9,929,206 discloses a device in which a plurality ofsingle-mode filters is two-dimensionally arrayed. The peak wavelengthsof the plurality of single-mode filters differ from one filter toanother. When the number of peaks in the transmission spectra of thefilter array in which the plurality of single-mode filters istwo-dimensionally arrayed is represented by N, the spatial resolution ofeach separate image decreases to 1/N even when a plurality of separateimages is reconstructed using the filter array. Accordingly, theoriginal images cannot be accurately reproduced with the separateimages, unlike the example shown in the middle stage in FIG. 8C. Thus,the device disclosed in U.S. Pat. No. 9,929,206 cannot provide anadvantage that is analogous to that of the light detecting device 300 inthe present embodiment.

Also, in the device disclosed in Japanese Unexamined Patent ApplicationPublication (Translation of PCT Application) No. 2015-501432, each of aplurality of sensors in a sensor array does not receive light with aplurality of wavelengths corresponding to peak wavelengths of amulti-mode filter and also does not reconstruct separate images 220 byusing information of the plurality of wavelengths. Accordingly, thedevice disclosed in Japanese Unexamined Patent Application Publication(Translation of PCT Application) No. 2015-501432 cannot provide anadvantage that is analogous to that of the light detecting device 300 inthe present embodiment.

Modifications of the light detecting device 300 shown in FIG. 5 will bedescribed next.

FIGS. 12A to 12F are views schematically showing modifications of thelight detecting device 300 shown in FIG. 5 .

As shown in FIG. 12A, in the filter array 100C, the filters 100 may bedivided. All the filters 100 do not have to be divided. Some of thefilters 100 may be divided.

As shown in FIG. 12B, the filters 100 do not have to be disposed abovesome of the pixels. In other words, in the filter array 100C, at leastone of the filters 100 may be transparent.

As shown in FIG. 12C, space may be provided between the filter array100C and the image sensor 60. In other words, the filter array 100C andthe image sensor 60 may be separated from each other with space beinginterposed therebetween.

As shown in FIG. 12D, one filter 100 may be disposed across a pluralityof pixels. In other words, the intermediate layer 26 may be continuouslyprovided across two or more pixels. The first reflective layer 28 aand/or the second reflective layer 28 b may be continuously providedacross two or more pixels.

As shown in FIGS. 12E and 12F, a transparent layer 27 may be disposed toplanarize level differences on the filter array 100C. In other words,the filter array 100C may further include a transparent layer 27 thatplanarizes level differences on two or more filters 100 having theabove-described resonance structures. In the example shown in FIG. 12E,level differences exist at upper surfaces of the second reflectivelayers 28 b in the filter array 100C. In the example shown in FIG. 12F,level differences exist at lower surfaces of the first reflective layers28 a in the filter array 100C. Planarizing level differences on two ormore filters 100 by using the transparent layer 27 facilitates thatanother member is disposed on the transparent layer 27.

As shown in FIGS. 12E and 12F, a plurality of microlenses 40 a may bedisposed on the filter array 100C. Each of the microlenses 40 a isdisposed on one of the filters 100. In other words, the filter array100C includes two or more microlenses 40 a. Each of the two or moremicrolenses 40 a is disposed on one of the two or more filters 100having the above-described resonance structures. Concentrating incidentlight by using the two or more microlenses 40 a makes it possible toefficiently detect light.

<Band-Pass Filter>

A band-pass filter may be used to pass some of a plurality of peakwavelengths in a transmission spectrum of the filter 100 and to removethe remainder. Redundant descriptions of those described above are notgiven hereinafter.

FIG. 13 is a view schematically showing a light detecting device 300 inanother exemplary embodiment. The light detecting device 300 furtherincludes a band-pass filter 29, in addition to the filter array 100C andthe image sensor 60. In the example shown in FIG. 13 , the band-passfilter 29 is disposed between the filter array 100C and the image sensor60 in the example shown in FIG. 5 . Herein, the filter array 100C andthe band-pass filter 29 may be collectively referred to as an “opticalfilter 110”. That is, the optical filter 110 includes the filter array100C and the band-pass filter 29.

In the filter array 100C, the transmission spectrum of each of thefilters 100 has local maximum values at three or more peak wavelengthsin a specific wavelength region of the image sensor 60. Each of thefilters 100 does not have to be a Fabry-Perot filter.

The band-pass filter 29 overlaps the entire filter array 100C whenviewed in a direction orthogonal to the filter array 100C. The band-passfilter 29 passes light with two or more peak wavelengths of three ormore peak wavelengths in the specific wavelength region. The band-passfilter 29 removes light with one or more peak wavelengths of the threeor more peak wavelengths in the specific wavelength region. Since lightwith two or more peak wavelengths is passed, an advantage of themulti-mode filters in the filter 100 is exploited.

In summary, the band-pass filter 29 passes light in a partial wavelengthregion included in the specific wavelength region of the image sensor60. The partial wavelength region includes two or more peak wavelengthsof the three or more peak wavelengths and do not include one or morepeak wavelengths of the three or more peak wavelengths.

Besides the example shown in FIG. 13 , each transmission spectrum of twoor more filters of the plurality of filters 100 may have local maximumvalues at three or more peak wavelengths in a specific wavelength regionof the image sensor 60. The band-pass filter 29 may overlap the two ormore filters when viewed in a direction orthogonal to the filter array100C. Of the plurality of filters 100, at least one filter other thanthe two or more filters may be transparent. Of the plurality of filters100, the transmittance of at least one filter other than the two or morefilters may have one or more local maximum values in a specificwavelength region.

Although, in the example shown in FIG. 13 , the filter array 100C andthe band-pass filter 29 are in contact with each other, they may beseparated from each other. Although, in the example shown in FIG. 13 ,the optical filter 110 and the image sensor 60 are in contact with eachother, they may be separated from each other.

An example of peak wavelength removal performed by the band-pass filter29 will be described next.

FIG. 14A is a diagram schematically showing an example of transmissionspectra of filters A, B, and C and an example of a transmission spectrumof the band-pass filter 29. In the example shown in FIG. 14A, thespecific wavelength region of the image sensor 60 corresponds to thetarget wavelength region W shown in FIG. 3A. The target wavelengthregion W is divided into wavelength regions W1 to W8. Filters A, B, andC each have three or more peak values at different wavelengths in thespecific wavelength region in the transmission spectrum. The numbers ofpeak values in filters A, B, and C are ten, nine, and eight,respectively. The band-pass filter 29 passes light with the wavelengthregions W3 to W6 in the specific wavelength region and removes lightwith the wavelength regions W1 to W2 and the wavelength regions W7 andW8.

FIG. 14B is a diagram schematically showing an example of a transmissionspectrum of one filter constituted by a set of each of filters A, B, andC and the band-pass filter 29. As shown in FIG. 14B, the transmissionspectrum has peak wavelengths in a partial wavelength region in thespecific wavelength region. The partial wavelength region corresponds tothe target wavelength region W shown in FIG. 3A. The target wavelengthregion W that is narrower than the specific wavelength region is dividedinto wavelength regions W1 to W4. The wavelength regions W1 to W4 shownin FIG. 14B respectively correspond to the wavelength regions W3 to W6shown in FIG. 14A.

When comparison is made between the examples shown in FIGS. 14A and 14B,the band-pass filter 29 reduces the target wavelength region W tothereby make it possible to reduce the number of regions divided in thetarget wavelength region W. This makes it possible to reduce the sizesof the matrix H and the vector f in expressions (1) and (2). As aresult, the signal processing circuit 200 shown in FIG. 1 can processimage signals output from the image sensor 60 with a small amount ofarithmetic operation load and can generate and output a plurality ofpieces of image data in a short period of time. Each of the plurality ofpieces of image data is image data for one of the plurality ofwavelength regions in the above-described partial wavelength region. Theplurality of pieces of image data represents a plurality ofreconstructed separate images 220.

The band-pass filter 29 may pass light with a plurality of discretewavelength regions in the specific wavelength region.

FIG. 15A is a diagram showing a modification of the example shown inFIG. 14A. The transmission spectrum of the band-pass filter 29 shown inFIG. 15A differs from the transmission spectrum of the band-pass filter29 shown in FIG. 14A. The band-pass filter 29 shown in FIG. 15A passeslight with wavelength regions W2 and W6 to W8 in the specific wavelengthregion and removes light with wavelength regions W1 and W3 to W5.

FIG. 15B is a diagram showing a modification of the example shown inFIG. 14B. As shown in FIG. 15B, the transmission spectrum has peakwavelengths in two discrete wavelength regions in the specificwavelength region. The two discrete wavelength regions correspond to thetarget wavelength region W shown in FIG. 3A. The target wavelengthregion W that is narrower than the specific wavelength region is dividedinto wavelength regions W1 to W4. Of the two discrete wavelengthregions, one wavelength region is the wavelength region W1, and theother wavelength region corresponds to the wavelength regions W2 to W4.The wavelength regions W1 to W4 shown in FIG. 15B correspondrespectively to the wavelength regions W2 and W6 to W8 shown in FIG.15A. Even in this case, separate images 220 can be reconstructed in ashort time. Of the plurality of peak wavelengths shown in FIG. 15A, oneor more peak wavelengths that are not included in the target wavelengthregion W shown in FIG. 15B can also be said to be located between two ormore peak wavelengths included in the target wavelength region W.

Next, modifications of the light detecting device 300 shown in FIG. 13will be described.

FIGS. 16A to 16D are views schematically showing modifications of thelight detecting device 300 shown in FIG. 13 .

As shown in FIG. 16A, the transparent layer 27 may be disposed betweenthe band-pass filter 29 and the image sensor 60 shown in FIG. 13 . Asshown in FIG. 16A, the optical filter 110 may include the filter array100C, the band-pass filter 29, and the transparent layer 27.

In the filter array 100C shown in FIG. 13 , the filters 100 may also bedivided, as shown in FIG. 16B.

Level differences on the filter array 100C shown in FIG. 13 may beplanarized with the transparent layer 27, as shown in FIG. 16C. Theband-pass filter 29 may be disposed on the transparent layer 27. Theplurality of microlens 40 a may be disposed on the band-pass filter 29.As shown in FIG. 16C, the optical filter 110 may include the filterarray 100C, the band-pass filter 29, and the transparent layer 27. Theoptical filter 110 may further include the plurality of microlens 40 a.

As shown in FIG. 16D, the band-pass filter 29 disposed on thetransparent layer 27 and the filter array 100C disposed on the imagesensor 60 may be separated from each other. As shown in FIG. 16D, theoptical filter 110 may include the filter array 100C, the band-passfilter 29, and the transparent layer 27.

In addition, the band-pass filter 29 may be disposed in or on each ofthe configurations shown in FIGS. 12A to 12F, to configure a lightdetecting device 300 other than the examples shown in FIGS. 16A to 16D.Also, the geometric relationship between the filter array 100C and theband-pass filter 29 is as follows. The above-described two or morefilters 100 in the filter array 100C may be disposed on an optical pathof light that passes through the band-pass filter 29. Conversely, theband-pass filter 29 may be disposed on an optical path of light thatpasses through the above-described two or more filters 100 in the filterarray 100C.

The optical filter, the light detecting device, and the light detectingsystem in the present disclosure are useful for, for example, camerasand measurement equipment that acquire multi-wavelength two-dimensionalimages. The optical filter, the light detecting device, and the lightdetecting system in the present disclosure can also be applied tosensing for living bodies, medical care, and beauty care, inspectionsystems for foreign matter and residual agricultural chemicals in food,remote sensing systems, vehicle-mounted sensing systems, and so on.

What is claimed is:
 1. An optical filter comprising: a filter array including a plurality of filters that is two-dimensionally arrayed, the plurality of filters including a first filter and a second filter, a transmission spectrum of each of the first filter and the second filter having local maximum values of transmittance at three or more wavelengths included in a first wavelength region; and a band-pass filter that passes light in a second wavelength region including two or more wavelengths of the three or more wavelengths and not including one or more wavelengths of the three or more wavelengths, wherein the filter array and the band-pass filter are disposed so that (a) the band-pass filter is located on an optical path of transmitted light that passes through the first filter and the second filter or (b) the first filter and the second filter are located on an optical path of transmitted light that passes through the band-pass filter.
 2. The optical filter according to claim 1, wherein each of the first filter and the second filter includes a first reflective layer, a second reflective layer, an intermediate layer between the first reflective layer and the second reflective layer and has a resonance structure having a plurality of resonant modes, orders of the plurality of resonant modes being different from each other; at least one selected from the group consisting of a refractive index and a thickness of the intermediate layer in the first filter differs from the at least one selected from the group consisting of a refractive index and a thickness of the intermediate layer in the second filter; the plurality of resonant modes includes three resonant modes; and the three or more wavelengths correspond to the three resonant modes, respectively.
 3. The optical filter according to claim 2, wherein each of the plurality of filters has the resonance structure.
 4. The optical filter according to claim 1, wherein the filter array further includes a transparent layer for planarizing a level difference between a surface of the first filter and a surface of the second filter.
 5. The optical filter according to claim 1, wherein at least one of the plurality of filters is transparent.
 6. The optical filter according to claim 1, wherein the plurality of filters includes a third filter that is different from the first filter and the second filter, and a transmission spectrum of the third filter has one or more local maximum values of transmittance.
 7. The optical filter according to claim 1, wherein the filter array is separated from the band-pass filter.
 8. The optical filter according to claim 1, wherein the filter array is in contact with the band-pass filter.
 9. The optical filter according to claim 1, wherein the band-pass filter overlaps all of the filter array when viewed in a direction orthogonal to the filter array.
 10. The optical filter according to claim 1, wherein the one or more wavelengths not included in the second wavelength region are located between a longest wavelength and a shortest wavelength of the two or more wavelengths included in the second wavelength region.
 11. A light detecting device comprising: the optical filter according to claim 1; and an image sensor including a plurality of light detection elements, each having sensitivity to light in the first wavelength region.
 12. The light detecting device according to claim 11, wherein the optical filter is in contact with the image sensor.
 13. The light detecting device according to claim 11, wherein the optical filter is separated from the image sensor.
 14. The light detecting device according to claim 11, further comprising optical system including at least one lens, wherein the optical system, the optical filter and the image sensor are arranged in this order such that light enters from the optical system toward the image sensor.
 15. A light detecting system comprising: the light detecting device according to claim 11; and a signal processing circuit, wherein, based on a signal output from the image sensor, the signal processing circuit generates and outputs image data including information of a plurality of wavelengths included in the second wavelength region.
 16. The light detecting system according to claim 15, wherein the image data includes data representing images spectrally separated for the plurality of wavelengths.
 17. The light detecting system according to claim 16, wherein the number of picture elements included in each of the images is larger than M/N, where N represents the number of images, and M represents the number of light detection elements.
 18. The light detecting system according to claim 17, wherein the number of picture elements is equal to the number of light detection elements. 